1. Field of the Invention
The present invention generally relates to a shake-free image restoration system and, more particularly, to a shake-free image restoration apparatus capable of producing steady and clear images or pictures as final outputs without installing an image correction device in the main body of a camera or other apparatus for shooting scenes incorporating the same. It also relates to a camera capable of restoring images from shakes on the basis of the data obtained for the shaking condition of the camera during film exposure by shake detecting and image restoring means incorporated in the camera main body.
The present invention further relates to an image restoration apparatus for restoring still pictures or images on films involving blurs caused by a shake and to regenerate them by compensating the blurs.
2. Description of the Related Art
A number of methods have been proposed to correct shaken and blurred picture images taken by cameras, including the following.
Firstly, "Youich Miyake: A Recommendable Way for image Processing, Shashin Kogyo, 109-113, 6/1987" describes a method of correcting a blurred image of an uniformly moving object, which can be summarized as follows. If an object f(x, y) being taken by a camera is moving in direction x at speed V for T seconds and the camera records an image g(x, y) of the moving object, then the relationship between the object and the image will be expressed by equation (1) below. ##EQU1##
when both sides are put to a Fourier transformation, then equation (2) below will be obtained. ##EQU2##
Differently stated, the following relationship, or equation (3) , will hold true. EQU H(u, v)=sin .pi.uVt/.pi.uV=T sin C(uVT) (3)
Then, by an inverse Fourier transformation of equation (4) below, the image of the object blurred by its motion will be restored. EQU F(u, v)=G(u, v)/H(u, v) (4)
Secondly, Published Unexamined Japanese Patent Application No. 5-72592 discloses a method of correcting blurred images by driving the film by novel drive means.
Finally, Published Unexamined Japanese Patent Application No. 63-187883 discloses an invention of an image pick-up apparatus capable of processing the picked-up image, using the data collected on the shaking condition of the apparatus during the image pick-up operation.
An apparatus for detecting shaking conditions of an image pick-up apparatus during the image pick-up operation and correcting the obtained blurred image to put it back to a clear image by driving film or optical means such as lenses is required to accurately detect any shaking conditions of the camera and carry out quick and accurate image correcting operations.
An apparatus that meets these and other requirements will therefore inevitably be very large, although such a large apparatus would not be able to satisfactorily correct blurred images because its function is based on an assumption that the object is moving in a uniform motion whereas moving objects normally show a not uniform but random motion.
There have also been proposed techniques for recording the extent of the shake by hand of a camera observed while it is used for taking a picture, notifying the camera user of the recorded data and/or correcting the obtained blurred image by using the data. A sensor such as an acceleration sensor or an angular velocity sensor is used for the detection of shaking motion of a camera.
The above cited Published Unexamined Japanese Patent Application No. 63-187883 also discloses a technique of detecting the physical quantity of the shake by hand of a camera and recording the obtained data on a recording medium.
However, an acceleration sensor or an angular velocity sensor is hardly capable of providing correct data on the shake of a camera while it is being used for taking a picture because any attempt to improve the S/N ratio of the signal representing the data requires the use of one or a plurality of filters, which in turn delays the timing of signal transmission.
Moreover, since the technique disclosed in the above Published Unexamined Japanese Patent Application No. 63-187883 does not pay attention to the delay in the timing of signal transmission of such a sensor, it does not and cannot provide correct data on the shake of the camera that influences the picture being taken.
There have also been known image processing apparatuses capable of performing a photoelectric conversion for the image picked-up by an image pick-up apparatus and arithmetically further processing the image by means of a processing apparatus such as a computer. Various techniques have also been proposed for restoring pictures that are blurred due to a shake by hand of cameras when they are taken. With such techniques, a single spot is selected on a blurred picture and the penetration and expansion of the spot into the surrounding area is mathematically expressed so that the blur of the picture may be mathematically handled.
For instance, Published Unexamined Japanese Patent Application No. 62-127976 discloses a method of establishing mathematical functions respectively having the effects of offsetting "out of focus", "shake" and "aberration", applying them to picture signals of the obtained picture by means of a photoelectric transducer device and arithmetically processing them to restore the picture.
Published Unexamined Japanese Patent Application No. 3-159482 teaches a method of using picture signals and inverse matrixes of divergence functions for blurred pictures to restore the pictures that are free from blurs.
A method is also known of performing a Fourier transformation on picture data and divergence functions so that pictures may be corrected in a Fourier space to restore the pictures by Fourier transformation.
However, divergence functions for pictures blurred by a shake by hand of a camera is one-dimensional unlike those for pictures with out of focus images or those pictures carrying the phenomenon of aberration. In other words, when out of focus or aberration is involved, the image of a spot in a picture expands two-dimensionally to blur the picture, whereas the image of a spot expands in a picture so as to draw a trace of a motion to consequently make the picture unclear if a shake is involved.
While it has been known that a shake by hand of a camera typically has a frequency of less than several Hz at the position of the image on the camera, the shake appears as lines on the picture taken by the camera if the duration of exposure is several tenths seconds. In other words, since a shake by hand of a camera normally results in a linear trace of a motion for any spot in the picture, it will be safe to say that each of the divergence functions expressing the shake of a picture represents a curved line or, very probably, a straight line. (See FIGS. 21A, B and C.)
Thus, when reading out a picture by means of a photoelectric transducer device, the data for the picture take the form of pixels, each representing the data for a tiny discrete spot in the picture. So far as the divergence functions involved are generated uniformly and two-dimensionally, the orientation of pixel arrangement does not have any particular significance and similar data will be obtained for any orientation.
When, however, the divergence functions involved in an operation of processing a picture are one-dimensional, the picture data for the picture varies depending on the orientation of the matrix of pixels, meaning that the divergence functions to be used for restoring the picture can be inappropriate and useless for the picture data depending on the orientation. This is because, if the picture carries a linear trace of a motion of a spot which is slanted relative to the matrix of pixels, the data for the motion cannot be linearly arranged. In other words, if the matrix of pixels of a picture has a direction of arrangement different from the direction along which a motion of a spot occurs in the picture, the obtained picture data inevitably involves blurs given rise to by causes other than the shake and, therefore, the picture cannot be restored simply by means of the divergence functions for the shake.
Additionally, since the obtained picture data can contain data representing an area outside the boundary of the picture located at and near an end of the picture opposite to the direction along which the camera is moved and such data are indefinable for Fourier transformation if the divergence functions for the shake are used for a matrix, consequently, the picture is short of satisfactory restoration.